feat(sketch_db): sid-level timeline_for_metric - #183
Merged
Merged
Conversation
Schema retirement #1 of 5. Adds `sketch_db::query::timeline::timeline_for_metric(&SketchStore, metric, t1_ms, t2_ms) -> Vec<TimelineSegment>` that computes the per-metric historical timeline entirely from the sid catalog (`SketchStore::instances`). Algorithm: 1. Snapshot all `SketchInstanceMetadata` for `metric` from the sid catalog. 2. Group by content signature `(metric, agg_kind, group_by_keys)`. Multiple sids sharing the same agg-config fold into one group. 3. Per group, fold the lifecycle fields: `min(first_seen_unix_ms)`, `Some(min(retired_at_ms))` iff every sid is retired, status = Active > Retired > Expired. 4. Apply the same segmenting + clipping as `SchemaRegistry::timeline_for_metric`. `TimelineSegment.agg_id` now carries a stable xxh64 of the content signature (the `(metric, agg_kind, group_by_keys)` tuple) — same content-derived id idiom as `compute_agg_config_id` (PR #151). HTTP callers see deterministic ids that don't depend on which specific sid was first seen. 7 new unit tests cover: empty store, inverted range, single-active signature, two-signatures-in-sequence, fold-of-many-sids-same-sig, metric isolation, and signature-id determinism. Schema/'s implementation stays alive in parallel until the consumers migrate. Adds `SketchStore::snapshot_instances()` to support read-side scans. 783 + 7 = 790 lib tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 task
zzylol
added a commit
that referenced
this pull request
May 13, 2026
Schema retirement #2 of 5. The `/api/v1/db/timeline` handler now reads from `state.sketch_index` (always attached) via `sketch_db::query::timeline::timeline_for_metric` (PR #183) instead of `state.schemas.timeline_for_metric`. User-visible behavior: - Endpoint always works (no more 503 when schema-registry isn't wired — the sid catalog is always present). - `agg_id` field carries a stable content-derived signature id (xxh64 of `metric + agg_kind + group_by_keys`) instead of the controller-emitted `agg_id` (which is gone after M2.2 / PR #152). - Segments now reflect the sid catalog directly; reconfigure semantics propagate once the next sub-PR (lifecycle reconcile) lands. Test impact: - `test_get_timeline_without_registry_returns_503` rewritten to `..._with_no_sids_returns_empty_200` matching the new semantics. - `test_get_timeline_returns_segments_after_reconfigure` ignored (uses POST /streaming-config → SchemaRegistry::reconcile, which doesn't yet propagate to the sid catalog; re-enabled in the next sub-PR with the lifecycle reconcile). 789/792 lib tests pass (3 ignored — 2 pre-existing + this one). Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2 tasks
zzylol
added a commit
that referenced
this pull request
May 13, 2026
#185) Schema retirement #3. Repoint `ASAPQueryEngine::timeline_for_query` from `SchemaRegistry::timeline_for_metric` to the sid-level `storage_engines::sketch_db::query::timeline::timeline_for_metric` landed in #183. The cross-reconfigure dispatcher (`try_handle_query_promql_via_timeline`) now reads its segments from the sid catalog rather than from `SchemaRegistry`. The sid-level timeline populates `TimelineSegment.agg_id` with a content-hash of `(metric, agg_kind, group_by_keys)` rather than a `StreamingConfig.aggregation_id`. Until schema retirement #5 ports the per-segment dispatch to sid-level evaluation, the segment-→-aggregation_config lookup inside the dispatcher is best-effort: when no segment resolves to an in-config aggregation the dispatcher returns `None` so the caller falls back to the default single-agg path instead of regressing cross-reconfigure queries to empty-result-plus-warnings. The schema retirement plan keeps the `schema_registry` field on `ASAPQueryEngine` alive for now — it's still referenced by the ingest barrier and the swap-handler driver. Both go away in retirements #4 + #5. Two tests in `tests/schema_timeline_dispatch_tests.rs` are marked `#[ignore]`: they build two distinct `AggregationConfig`s with identical content (same metric / Sum / `host` grouping). In the sid catalog those collapse to one signature group → one segment, so the dispatcher can no longer reproduce the schema-boundary-stitch scenario from a SchemaRegistry-shaped fixture. The third single-schema regression test still passes unchanged. Re-enabling these is part of retirement #5 (sid-level dispatch) or a fixture rewrite that uses two genuinely distinct signatures. 787/787 lib tests pass; 5 ignored. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Schema retirement #1 of 5. Adds
sketch_db::query::timeline::timeline_for_metric(&SketchStore, metric, t1, t2) -> Vec<TimelineSegment>computed entirely from the sid catalog.Schema/'s
SchemaRegistry::timeline_for_metricstays alive in parallel; this PR just adds the sid-level version. Consumer migration (HTTP/api/v1/db/timeline, engine cross-reconfigure dispatch) is the next sub-PR.Algorithm
SketchStore.instancesformetric.(metric, agg_kind, group_by_keys). Many sids of one agg-config → one group.min(first_seen_unix_ms), retired-iff-all-retired, status precedence Active > Retired > Expired.SchemaRegistry::timeline_for_metric.TimelineSegment.agg_idcarries a stable xxh64 of the content signature — same content-derived id idiom ascompute_agg_config_id(PR #151).Test plan
sketch_db::query::timeline::tests.cargo test -p data_plane --lib— 790/790 pass.What's next
/api/v1/db/timelineconsumer.lifecycle/).schema/.🤖 Generated with Claude Code